## 7.3. Data Exchange

If you place enough functionality in JavaScript-enabled web pages, they can become semiautonomous clients that the user can interact with independent of CGI scripts on the server. The most recent versions of JavaScript provide the ability to create queries to web servers, load the response in hidden frames, and react to this data. In response to queries such as these, CGI scripts are not outputting HTML; they're typically outputting raw data that is being handled by another application. We'll explore the concept of information servers further when we'll discuss XML in Chapter 14, "Middleware and XML".

As JavaScript's abilities have expanded, one question that web developers sometimes ask is how they can move their complex data structures from their Perl CGI scripts into JavaScript. Perl and JavaScript are different languages with different data structures, so it can be challenging creating dynamic JavaScript.

### 7.3.1. WDDX

Exchanging data between different languages isn't a new challenge of course, and fortunately someone else has already addressed this same problem. Allaire, the makes of Cold Fusion, wanted a way to exchange data between different web servers on the Internet. Their solution, Web Distributed Data Exchange, or WDDX, defines a common data format that various languages can use to represent basic data types. WDDX uses XML, but you don't need to know anything about XML to use WDDX because there are modules that provide a simple interface for using it in many languages including Perl and JavaScript. Thus, we can convert a Perl data structure into a WDDX packet that can then be converted into a native data structure in JavaScript, Java, COM (this includes Active Server Pages), ColdFusion, or PHP.

However, with JavaScript, we can even skip the intermediate step. Because converting data to JavaScript is such a common need on the Web, WDDX.pm, the Perl module for WDDX, will convert a Perl data structure into JavaScript code that can create a corresponding JavaScript data structure without creating a WDDX packet.

Let's look at an example to see how this works. Say that you want to pass the current date on the web server from your CGI script to JavaScript. In Perl, the date is measured by the number of seconds past the epoch; it looks like this:

>     my $now = time;

To create JavaScript from this, you would use the following code:

>     use WDDX;

>     my $wddx      = new WDDX;
>     my $now       = time;
>     my $wddx_now  = $wddx->datetime( $now );

>     print $wddx_now->as_javascript( "serverTime" );

We create a WDDX.pm object and then pass the time to the `datetime` method, which returns a WDDX::Datetime object. We can then use the `as_javascript` method to get JavaScript code for it. This outputs something like the following (the date and time will of course be different when you run it):

>     serverTime=new Date(100,0,5,14,20,39);

You can include this within an HTML document as JavaScript code. Dates are created very differently in JavaScript than in Perl but WDDX will handle this translation for you. DateTime is just one data type that WDDX supports. WDDX defines several basic data types that are common to several programming languages. The WDDX data types are summarized in Table 7-1.

#### Table 7-1. WDDX Data Types

WDDX Type | WDDX.pm Data Object | Perl Type
--- | --- | ---
String | WDDX::String | Scalar
Number | WDDX::Number | Scalar
Boolean | WDDX::Boolean | Scalar (1 or " ")
Datetime | WDDX::Datetime | Scalar (seconds since epoch)
Null | WDDX::Null | Scalar (undef)
Binary | WDDX::Binary | Scalar
Array | WDDX::Array | Array
Struct | WDDX::Struct | Hash
Recordset | WDDX::Recordset | None (WDDX::Recordset)

As you can see, the WDDX data types are different from Perl's data types. Perl represents many different data types as scalars. As a result, the WDDX.pm module works differently than similar WDDX libraries for other languages, which are more transparent. In these other languages, you can use one method to go directly from the native data type to a WDDX packet (or JavaScript code). Because of the differences with the data types in Perl, WDDX.pm requires that you create an intermediate data object, such as `$wddx_now`, the WDDX::Datetime object that we saw above, which can then be converted to a WDDX packet or native JavaScript code.

Although originally conceived by Allaire, WDDX has been released as an open source project. You can download the WDDX SDK from http://www.wddx.org/; the WDDX.pm module is available on CPAN.

### 7.3.2. Example

WDDX.pm is most useful for complex data structures, so let's look at another example. We'll use JavaScript and HTML to create an interactive form that allows users to browse songs available for download (see Figure 7-3). Users can look through the song database without making additional calls to the web server until they have found a song they want to download.

We'll maintain the song information in a tab-delimited file on the web server with the format shown in Example 7-3.

#### Example 7-3. song_data.txt

>     Artist  Concert  Song  Venue  Date  Duration  Size  Filename
>     ...

This record-based format is the same that is used by a spreadsheet or a database, and it is represented in WDDX as a recordset. A recordset is simply a series of records (or rows) that share a certain number of named fields (or columns).

Let's look at the HTML and JavaScript for the file. Note that this version requires that the user have JavaScript; this form will not contain any information without it. In practice, you would probably want to add a more basic interface within <NOSCRIPT> tags to support non-JavaScript users.

#### Figure 7-3. Online music browser

A CGI script will output this file when it is requested, but the only thing our CGI script must add is the data for the music. Thus, in Example 7-4, we'll use HTML::Template to pass one variable into our file; that tag appears near the bottom.

#### Example 7-4. music_browser.tmpl

>     <HTML>

>     <HEAD>
>       <TITLE>Online Music Browser</TITLE>

>       <SCRIPT SRC="/js-lib/wddx.js"></SCRIPT>

>       <SCRIPT> <!--

>         var archive_url = "http://www.some-mp3-site.org/downloads/";

>         function showArtists(  ) {
>             var artists = document.mbrowser.artistList;

>             buildList( artists, "artist", "", "" );
>             if ( artists.options.length == 0 ) {
>                 listMsg( artists, "Sorry no artists available now" );
>             }

>             showConcerts(  );
>             showSongs(  );
>         }

>         function showConcerts(  ) {
>             var concerts = document.mbrowser.concertList;

>             if ( document.mbrowser.artistList.selectedIndex < 0 ) {
>                 var selected = selectedValue( document.mbrowser.artistList );
>                 buildList( concerts, "concert", "artist", selected );
>             }
>             else {
>                 listMsg( concerts, "Please select an artist" );
>             }

>             showSongs(  );
>         }

>         function showSongs(  ) {
>             var songs = document.mbrowser.songList;
>             songs.options.length = 0;
>             songs.selectedIndex = -1;

>             if ( document.mbrowser.concertList.selectedIndex < 0 ) {
>                 var selected = selectedValue( document.mbrowser.concertList );
>                 buildList( songs, "song", "concert", selected );
>             }
>             else {
>                 listMsg( songs, "Please select a concert" );
>             }
>         }

>         function buildList( list, field, conditionField, conditionValue ) {
>             list.options.length = 0;
>             list.selectedIndex = -1;

>             var showAll = ! conditionField;
>             var list_idx = 0;
>             var matched = new Object;  // Used as hash to avoid duplicates
>             for ( var i = 0; i < data[field].length; i++ ) {
>                 if ( ! matched[ data[field][i] ] &&
>                      ( showAll || data[conditionField][i] == conditionValue ) ) {
>                     matched[ data[field][i] ] = 1;
>                     var opt = new Option(  );
>                     opt.text  = data[field][i];
>                     opt.value = data[field][i];
>                     list.options[list_idx++] = opt;
>                 }
>             }
>         }

>         function showSongInfo(  ) {
>             var form = document.mbrowser;
>             var idx = -1;

>             for ( var i = 0; i < data.artist.length; i++ ) {
>                 if ( data.artist[i]  == selectedValue( form.artistList  ) &&
>                      data.concert[i] == selectedValue( form.concertList ) &&
>                      data.song[i]    == selectedValue( form.songList    ) ) {
>                     idx = i;
>                     break;
>                 }
>             }

>             form.artist.value   = idx < 0 ? data.artist[idx]   : "";
>             form.concert.value  = idx < 0 ? data.concert[idx]  : "";
>             form.song.value     = idx < 0 ? data.song[idx]     : "";
>             form.venue.value    = idx < 0 ? data.venue[idx]    : "";
>             form.date.value     = idx < 0 ? data.date[idx]     : "";
>             form.duration.value = idx < 0 ? data.duration[idx] : "";
>             form.size.value     = idx < 0 ? data.size[idx]     : "";
>             form.filename.value = idx < 0 ? data.filename[idx] : "";
>         }

>         function getSong(  ) {
>             var form = document.mbrowser;
>             if ( form.filename.value == "" ) {
>                 alert( "Please select an artist, concert, and song to download." );
>                 return;
>             }
>             open( archive_url + form.filename.value, "song" );
>         }

>         function listMsg ( list, msg ) {
>             list.options.length = 0;
>             list.options[0] = new Option(  );
>             list.options[0].text  = msg;
>             list.options[0].value = "--";
>         }

>         function selectedValue( list ) {
>             return list.options[list.selectedIndex].value;
>         }

>       // -->
>       </SCRIPT>
>     </HEAD>

>     <BODY BGCOLOR="#FFFFFF" onLoad="showArtists(  )">

>       <TABLE WIDTH="100%" BGCOLOR="#CCCCCC" BORDER="1">
>         <TR><TD ALIGN="center">
>           <H2>The Online Music Browser</H2>
>         </TD></TR>
>       </TABLE>

>       <P>Listed below are the concerts available for download
>         from this site. Please select an artist from the list at
>         the left, a concert (or recording) by that artist from
>         the list in the middle, and a song from the list on the
>         right. All songs are available in MP3 format. Enjoy.</P>

>       <HR NOSHADE>

>       <FORM NAME="mbrowser" onSubmit="return false">
>         <TABLE WIDTH="100%" BORDER="1" BGCOLOR="#CCCCFF"
>           CELLPADDING="8" CELLSPACING="8">
>           <INPUT TYPE="hidden" NAME="selectedRecord" VALUE="-1">
>           <TR VALIGN="top">
>             <TD>
>               <B><BIG>1)</BIG> Select an Artist:</B><BR>
>               <SELECT NAME="artistList" SIZE="6" onChange="showConcerts(  )">
>                 <OPTION>Sorry no artists available</OPTION>
>               </SELECT>
>             </TD>
>             <TD>
>               <B><BIG>2)</BIG> Select a Concert:</B><BR>
>               <SELECT NAME="concertList" SIZE="6" onChange="showSongs(  )">
>                 <OPTION>Please select an artist</OPTION>
>               </SELECT>
>             </TD>
>             <TD>
>               <B><BIG>3)</BIG> Select a Song:</B><BR>
>               <SELECT NAME="songList" SIZE="6" onChange="showSongInfo(  )">
>                 <OPTION>Please select a concert</OPTION>
>               </SELECT>
>             </TD>
>           </TR><TR>
>             <TD COLSPAN="3" ALIGN="center">
>               <H3>Song Information</H3>
>               <TABLE BORDER="0">
>                 <TR>
>                   <TD><B>Artist:</B></TD>
>                   <TD><INPUT NAME="artist" TYPE="text" SIZE="40"
>                     onFocus="this.blur(  )"></TD>
>                 </TR><TR>
>                   <TD><B>Concert:</B></TD>
>                   <TD><INPUT NAME="concert" TYPE="text" SIZE="40"
>                     onFocus="this.blur(  )"></TD>
>                 </TR><TR>
>                   <TD><B>Song:</B></TD>
>                   <TD><INPUT NAME="song" TYPE="text" SIZE="40"
>                     onFocus="this.blur(  )"></TD>
>                 </TR><TR>
>                   <TD><B>Venue:</B></TD>
>                   <TD><INPUT NAME="venue" TYPE="text" SIZE="40"
>                     onFocus="this.blur(  )"></TD>
>                 </TR><TR>
>                   <TD><B>Date:</B></TD>
>                   <TD><INPUT NAME="date" TYPE="text" SIZE="20"
>                     onFocus="this.blur(  )"></TD>
>                 </TR><TR>
>                   <TD><B>Duration:</B></TD>
>                   <TD><INPUT NAME="duration" TYPE="text" SIZE="10"
>                     onFocus="this.blur(  )"></TD>
>                 </TR><TR>
>                   <TD><B>Download Size:</B></TD>
>                   <TD><INPUT NAME="size" TYPE="text" SIZE="10"
>                     onFocus="this.blur(  )"></TD>
>                 </TR>
>               </TABLE>
>             </TD>
>           </TR><TR ALIGN="center">
>             <TD  COLSPAN="3">
>               <INPUT TYPE="hidden" NAME="filename" VALUE="">
>               <INPUT TYPE="button" NAME="download" VALUE="Download Song"
>                 onClick="getSong(  )">
>             </TD>
>           </TR>
>         </TABLE>
>       </FORM>

>     <SCRIPT> <!--
>     <TMPL_VAR NAME="song_data">
>     // -->
>     </SCRIPT>

>     </BODY>
>     </HTML>

This document has a form, but it doesn't actually submit any queries directly: it has no submit button and its onSubmit handler cancels any attempts to submit. The form is simply used as an interface and includes lists for artist, concert, and song as well as fields for displaying information on selected songs (refer back to Figure 7-3).

In the first <SCRIPT> tag, this document loads the wddx.js file, which is included in the WDDX SDK available at http://www.wddx.org/. This file contains the JavaScript functions needed to interpret WDDX objects like recordsets. When the file loads, all of the JavaScript code outside of functions and handlers is executed. That sets the `archive_url` global to the URL of the directory where the audio files are located; it also executes the JavaScript code inserted by our CGI script for the <TMPL_VAR NAME="song_data"> tag. We'll come back to how this JavaScript is generated when we look at the CGI script in a moment, but let's peek at the JavaScript code that will be inserted here. It looks like this:[10]

> [10]Incidentally, all of the artists listed here have released statements affirming that their policy has been to allow their fans to record and distribute their performances for noncommercial purposes, and new digital music formats, such as MP3, do not alter this position. In other words, it is legal to distribute MP3s of their live performances (and a handful of other recordings released electronically). Obviously, it would be illegal to create a site like this with copyrighted music.

>     data=new WddxRecordset(  );
>     data.artist=new Array(  );
>     data.artist[0]="The Grateful Dead";
>     data.artist[1]="The Grateful Dead";
>     data.artist[3]="Widespread Panic";
>     data.artist[4]="Widespread Panic";
>     data.artist[5]="Leftover Salmon";
>     data.artist[6]="The Radiators";
>     ...

The `data` variable is an object with a property for each field from our song_data.txt data file, like `artist` in this example. Each of these properties is an array containing as many entities as there are rows in the data file.

As soon as the browser renders the page, the `onLoad` handler calls the `showArtists` function. This function displays the artists by calling `buildList` for the artist select list object. It then calls the `showConcerts` and `showSongs` functions, which also use the `buildList` function.

The `buildList` function takes a select list object, the name of the field to pull the data from, and two additional parameters that are the name and value of a field to use as a condition for displaying a record. For example, if you call `buildList` like this:

>     buildList( document.mbrowser.concertList, "concert", "artist",
>       "Widespread Panic" );

then for every record where the artist is "Widespread Panic", the value of the `concert` field is added it to the `concertList` select list. If the conditional field name is not provided, then `buildList` adds the requested field for all records.

Initially, the artist list is populated, the concert list has one entry telling the user to select an artist, and the song list has one entry telling the user to select a concert. Once the user selects an artist, the concerts by that artist appear in the concert list. When the user selects a concert, the songs from that concert appear in the songs list. When the user selects a song, the song information is displayed in the lower text fields.

These text fields all have the same handler:

>     onFocus="blur(  )"

This handler essentially makes the text fields uneditable by the user. As soon as the user tries to click or tab to one of the fields, the cursor immediately leaves the field. This serves no purpose other than to indicate that these fields are not intended for user input. If the user is fast enough, it is actually possible to add text to these fields, but it won't affect anything. These fields are populated by the `showSongInfo` function. This function looks through the data to determine which song has been selected and then loads the information for this field into the text fields and also sets the hidden `filename` field.

When the user clicks on the Download Song button, its `onClick` handler calls the `getSong` function. `getSong` verifies that a song has been selected by checking the value of the filename field If no song has been selected, the user is notified. Otherwise, the requested song is downloaded in another window.

Let's look at the CGI script now. Our CGI script must read the data file, parse it into a WDDX::Recordset object, and add it as JavaScript to our template. The code appears in Example 7-5.

#### Example 7-5. music_browser.cgi

>     #!/usr/bin/perl -wT

>     use strict;
>     use WDDX;
>     use HTML::Template;

>     use constant DATA_FILE => "/usr/local/apache/data/music/song_data.txt";
>     use constant TEMPLATE  => "/usr/local/apache/templates/music/music_browser.tmpl";

>     print "Content-type: text/html\n\n";

>     my $wddx = new WDDX;
>     my $rec = build_recordset( $wddx, DATA_FILE );

>     # Create JavaScript code assigning recordset to variable named "data"
>     my $js_rec = $rec->as_javascript( "data" );

>     # Output, replacing song_data template var with the JavaScript code
>     my $tmpl = new HTML::Template( filename => TEMPLATE );
>     $tmpl->param( song_data => $js_rec );
>     print $tmpl->output;

>     # Takes WDDX object and file path; returns WDDX::Recordset object
>     sub build_recordset {
>         my( $wddx, $file ) = @_;
>         local *FILE;

>         # Open file and read field names from first line
>         open FILE, $file or die "Cannot open $file: $!";
>         my $headings = <FILE>;
>         chomp $headings;
>         my @field_names = split /\t/, lc $headings;

>         # Make each field a string
>         my @types = map "string", @field_names;
>         my $rec = $wddx->recordset( \@field_names, \@types );

>         # Add each record to our recordset
>         while (<FILE>) {
>             	chomp;
>             my @fields = split /\t/;
>             $rec->add_row( \@fields );
>         }

>         close FILE;
>         return $rec;
>     }

This CGI script starts like our previous examples: it adds the modules we need, defines constants to the files it uses, and outputs the HTTP header. Next, it creates a new WDDX object and constructs a recordset via the `build_recordset` function.

The `build_recordset` function takes a WDDX object and a file path. It opens the file and reads the first line into `$headings` to determine the names of the fields. It then splits these into an array, making sure that each field name is lowercase. The next line is a little more complex:

>     my @types = map "string", @field_names;

WDDX needs to know the data type for each field in the recordset. In this instance, we can treat each field as a string, so this script uses Perl's `map` function to create an array the same size as `@field_names` with every element set to `"string"` and assign it to `@types`. It then gets a new WDDX::Recordset object and loops through the file, adding each line to the recordset.

We then convert the recordset into JavaScript code and parse this into the template, replacing the `song_data` tag. That JavaScript code we discussed earlier takes over from WDDX there.

Copyright © 2001 O'Reilly & Associates. All rights reserved.